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In this paper, we follow in the footsteps of Onsager and Machlup (OM) and consider diffusion¬ 
like paths that are explored by a particle moving via a conservative force while being in thermal 
equilibrium with its surroundings. Instead of considering diffusion (Brownian dynamics), we use a 
Metropolis algorithm to derive an OM-like functional. Through the lens of the Metropolis algorithm, 
we are able to elucidate the errors made when using a nonzero time increment. 

Of particular interest are transition paths that transverse an energy barrier that is large (but not 
too large) compared to the typical thermal energy. These transitions have probabilities that are 
only small and yet not so small as to be considered a violation of thermodynamics. As such, we 
turn our attention to the ’’double-ended” problem, where the OM functional can be interpreted as a 
’’thermodynamic” action and employed to sample paths that start and end at predetermined points. 
We find that sampling the continuous-time limit of the OM functional, i.e., the Ito-Girsanov form, 
produces unphysical paths due to thermodynamic inconsistency arising from the singular nature of 
the limit. These unphysical effects are due to the correlation between the particles position and 
the random fluctuating noise that is introduced in the limiting process. Such a correlation is a 
direct consequence of the form of the Ito-Girsanov functional, does not originate from numerical 
inaccuracies, and is a violation of thermodynamics as embodied by Seifert’s Integral Fluctuation 
Theorem. 

PACS numbers: 05.40.-a, 05.10.Gg, 05.40.Jc, 05.10.Ln 

INTRODUCTION 

The aim of this paper is much the same as the 1953 
article[T] of Onsager and Machlup, namely, to derive the 
probability of a succession of states of a spontaneously 
fluctuating thermodynamic system. Their expression 
for this (path) probability has become known as the 
Onsager-Machlup (OM) functional and has so far with¬ 
stood the test of time. One commentary even stated that 
the results of this 1953 paper are ’’incapable of improve¬ 
ment either in form or in their mode of derivation.” [5] 

Here we follow the spirit of the OM work, and try 
to fully understand its limitations while exploring the 
continuous-time limit. The derivation of the OM func¬ 
tional that we report here, has its roots in a different 1953 
article by Metropolis et al. [3] While the OM function can 
be derived by considering diffusion (in terms of Brown¬ 
ian dynamics), one can use the Metropolis algorithm to 
correct for any errors due to the nonzero time step. The 
strategy here is to use a variant, the Hybrid Monte Carlo 
(HMC) algorithm]!]. Using this approach, we derive an 
analogous OM functional and show how the functional 
is consistent with thermodynamics, that is, with the un¬ 
derlying Boltzmann distribution. 

The OM functionals used here act as a thermodynamic 
action]!] for generating paths that are constrained at 
both ends. One way to proceed is to take the continuous 
time limit of the functional; to use Ito calculus and the 
Girsanov theorem]!]. We have used this limiting func¬ 
tional to generate an ensemble of paths and found them 
to be unphysical. We show this unphysical nature orig¬ 


inates in the form of the Ito-Girsanov measure, which 
violates Seifert’s Integral fluctuation theorem]?]. 

BROWNIAN DYNAMICS 

Throughout this paper, we will consider a particle in 
contact with a heat reservoir at a temperature e. It is 
moving under the influence of a potential Y{x) with the 
force being F{x) = —Y’(x). Note that although the equa¬ 
tions are written for the one-dimensional case for clarity, 
the formalism can easily be extended to higher dimen¬ 
sions and for a collection of particles. 

The equation of motion for Brownian dynamics is given 
by 

dx = F(x) dt + dWt (1) 

where dWt is the standard Weiner process that repre¬ 
sents the (uncorrelated) Gaussian noise. Using a discrete 
time step, At, one typically uses the Euler-Maruyama 
algorithm]!] as an approximate method for propagating 
the position as a function of time. Namely, 

Xi+i = Xi + F{xi) At + V2eAt 

where ^ is a Gaussian random variate with mean zero and 
unit variance. Successive application (N times) of this 
equation produces a sequence of positions {xi} which is 
called a path. Onsager and Machlup]!] used the underly¬ 
ing thermal fluctuations to write the Gaussian probabil¬ 
ity, Pp oc H.i exp(—b^?), of the path in terms of the path 
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variables themselves, namely, 

i=l 

This equation is commonly called the OM functional. In 
the continuous-time limit, using Ito calculus and the Gir- 
sanov theorem, the Radon-Nikodym derivative is used to 
express the change in the measure: 

^ <=*> 

where T is the duration of the path, Q is the measure as¬ 
sociated with free Brownian dynamics, and the function 
G{x) is defined as G(x) = ^F{x) ■ F{x) — eY''{x). 

SOME RESULTS 

One of the uses of the OM functional is to incorporate 
it into a scheme to sample paths that are constrained 
at both ends. The aim is to efficiently generate an en¬ 
semble of paths that include a transition over an energy 
barrier. When the barrier is large compared to the typ¬ 
ical thermal energy, the transition is a rare event. Such 
barrier hopping is consistent with thermodynamics with 
the noise reflecting the fluctuating random effects that 
are independent of the particle’s position. This contrasts 
to what we designate as extremely rare events where an 
occurrence would seem to violate thermodynamics, for 
example when all the molecules in a room migrate to one 
corner. Here we only consider the former. 

Conventional forward time integration is not particu¬ 
larly efficient, especially for small e, since the hopping 
rate is small. We incorporate the Ito-Girsanov formula 
into a Monte Carlo method and sample the measure 
which, after some initialization, is used to generate paths 
that reflect the probability distribution. We use a re¬ 
cently devised path-space sampling method [5]. 

The quandary is that for a very simple one-dimensional 
example, the generated paths quickly become unphysi¬ 
cal. Long paths, generated with small time steps, are ex¬ 
pected to be consistent with equilibrium thermodynam¬ 
ics. The positions along the path should be distributed 
according to the Boltzmann probability, exp(—V/e). As 
an example, consider the potential 

V(a:) = 2-2®(8-5x)®(2-k5x)2 (4) 

which has two degenerate wells with a barrier of unity at 
the origin. A narrow (quadratic) well is on the left and 
the wide well is on the right. At any nonzero tempera¬ 
ture, the particle will spend more time on the right due 
to the entropic considerations inherent in the Boltzmann 
distribution. We used the Ito-Girasanov formulas with 



FIG. I: The fraction of the path in the broad well is 
shown as a function of an arbitrary sampling index, s. 
The results from both methods start with the same 
input path where B{0) « 0.6, have a path length of 
T = N At = 150, and a time step along the path of 
At = 0.005. The lower (red) curve corresponds to the 
results using the Ito-Girsanov form, while the upper 
(blue) curve are the results of using the mid-point 
functional. The (orange) dotted line corresponds to the 
equilibrium value. 

the method developed by Beskos et aim, at a temper¬ 
ature e = 0.25, to generate a sequence of these paths. 
We use the Heaviside function 0 to define B{s) as the 
fraction of the path that is contained in the broad well, 
namely 

Bis) = ^ [ dt 0(a;i^^) ~ 

Jo j 

where the sampling index is denoted as s, and the corre¬ 
sponding path is { In Figurej^ we plot i?(s) using 

the described procedure (red curve) and compare it to 
one using a second method described in a later section. 
As can be seen from the figure, the paths quickly become 
unphysical in that the particle spends the vast majority 
of the time in the left, thin well. Such paths are inconsis¬ 
tent with the equilibrium thermodynamical distribution. 
Thus the purpose of this article is to explain the origins of 
these unphysical results and to describe which functionals 
can be used to sample physical paths with double-ended 
boundary conditions. 

A NEW PERSPECTIVE 

Given the difficulty described above, we turn to an 
alternate derivation of the OM functional. In doing 
so, we expose where the errors are hidden in the tra¬ 
ditional view of Brownian dynamics. Instead of bas¬ 
ing a path-probability functional on Brownian dynam¬ 
ics, we will use the Metropolis-based, Hybrid Monte 
Garlo (HMC) methodjl]. In the incarnation used here, 
velocities are chosen from the temperature-appropriate 
Maxwell-Boltzmann (Gaussian) distribution. The de¬ 
terministic Hamilton’s equations are integrated over a 
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single time step h. The Metropolis acceptance step is 
determined by the change of the total energy that oc¬ 
curred during the deterministic integration. This proce¬ 
dure is sometimes called Metropolis Adjusted Langevin 
Algorithm (MALA) and Smart Monte Carlo[Tn]. Thus 
in an appropriate limit, the procedure has the random- 
walk properties of Brownian motion[TT|. Note that if the 
integration were exact, all moves would be accepted. 

The details are as follows. The configuration or posi¬ 
tion is Xi at time ti. The mass matrix is chosen to be 
the identity matrix. The velocities at the start of the 
interval are drawn from the Gaussian distribution with 
mean zero and variance set equal to the temperature. 
Then one integrates (approximately) Hamilton’s equa¬ 
tions over a time h. First we consider the leapfrog (or 
velocity-Verlet) scheme[T2]. The velocities at the begin¬ 
ning of the time interval are defined to be ^i- 

Then Xi+i is formed by 


target Boltzmann distribution. Using Brownian dynam¬ 
ics, without rejections, one samples a ’’near-by” distri¬ 
bution. The measures can be (in some sense) close to 
each other, but only if the forward integration of the 
SDE corresponds to a HMC method with a low rejection 
rate. The rejection rate is governed by the energy error 
(equation . 

Before continuing, it is instructive to re-express the 
OM functional for a path. The path is given by a se¬ 
quence of configurations, Xi. The path probability is 
given by f’path cx exp(—<i)/ 2 e) where the functional d) 
(up to an unimportant constant) is expressed as 


— =E f- 

At ^ \ 9 


i=l 
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F{xi-i) 


AE, 


( 8 ) 


At 


Xj+i =Xi + h uo,i -I- —F{xi). (5) 

The velocity at the end of the interval v is given by 

Vf,i = -k ^ (^F{xi) + F(xi+i) ^. (6) 

The error in the energy due to using this approximate 
quadrature is 


AEi = i (xi - Xi_i) • (^F(xj) -I- F(xj_i)^ 

+ V(xO-V(x,_i) + ^ (|F(xOp- |F(x,_i)p). 


( 7 ) 


The proposed position Xi+i should be accepted or 
rejected using the Metropolis-Hastings [13] test: If 
exp(—AAi/e) > 77 , the proposed move x^+i is accepted, 
otherwise Xi is reused. Here 77 G K is a random number 
uniformly chosen on the unit interval, 0 < rj < 1. The ac¬ 
ceptance probability is given by the Metropolis function 
M(Aifi/e) = Min(l,exp(—AAi/e)). Note that without 
rejections, detailed balance is broken. 


CONNECTION TO BROWNIAN DYNAMICS 

The paths (sequence of configurations) generated by 
the HMC method are similar to those generated by iter¬ 
ating the stochastic differential equation (SDE) of Brow¬ 
nian motion. This can be seen by identifying At = /2 

in equation This identification is consistent with the 
recent work of Spivak et aim). However, to fully recover 
the Brownian SDE, the rejection step of the Monte-Carlo 
algorithm must be ignored. This leads to a sampling er¬ 
ror that will be small if the step size is small. 

Thus the two methods sample different measures. The 
HMC method, with rejections, is designed to sample the 


where the A’s refer to changes as the position changes 
from Xi_i to Xi and AEi is defined in equation]^ The 
first term in the functional $ becomes the free Brown¬ 
ian measure (in the appropriate limit), and the fourth 
becomes —eY" when the quadratic variation (Ax^ « 
2 e At) is satisfied. The difference in potential energy 
telescopes, and the functional includes all components of 
the continuous time OM functional (equation]^. Below 
we show the importance of the energy error terms, AEi. 
It is well-known that the leapfrog integrator does not give 
accurate paths[T5|. This can be seen here as well, since 
the error in the path is of order one due to this term, and 
the energy error does not vanish even for a quadratic po¬ 
tential (an Ornstein-Uhlenbeck (OU) process [TB]1. 

It is important to recognize that rejections must be ig¬ 
nored to map the HMC method onto Brownian dynam¬ 
ics. Since the energy errors, in general, do not vanish, 
detailed balance is broken in the Metropolis procedure. 
The damage to the sampling ensemble is hard to quantify. 
In the Brownian picture, the energy-error terms leads to 
spurious entropy production as time-reversal invariance 
is broken. As the time step decreases, so do these errors. 

Note that Seifert’s Integral Fluctuation Theorem 
(IFT )|3 states that all paths (of the same duration) are 
equally probable. This is seen in equation by not¬ 
ing that logPp is simply proportional to the variance of 
the Gaussian fluctuations and thus is independent of the 
force, F. We see this in the HMC algorithm, where the 
choice of the velocities is independent of the particle po¬ 
sition and is the source of the stochastic variation. The 
noise can only be a function of the heat reservoir if one is 
to recover the thermodynamic (Boltzmann) distribution. 
When using a functional to generate double-ended paths, 
any functional that includes a correlation between the po¬ 
sitions and the velocities will produce paths that will be 
inconsistent with the Boltzmann distribution. Only in 
extremely rare instances will the noise be systematically 
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correlated with the particle position. Such instances are 
those events that are so improbable that they can be 
viewed as a violation of thermodynamics. 

In particular, while inspecting the Ito-Girsanov mea¬ 
sure, we note that some paths have higher probability 
than others. This violates Seifert’s Integral fluctuation 
theorem[7]. As one explores the paths distributed accord¬ 
ing to this measure, the sampling procedure will have a 
tendency to move to the region where the measure indi¬ 
cates it to be more probable. In those regions, the noise is 
highly correlated with the particle positions. Thus by its 
very form, the Ito-Girsanov OM functional must produce 
an ensemble of paths that will not be consistent with the 
Boltzmann distribution. 


THE MIDPOINT INTEGRATOR 

We can produce better functionals by improving the 
accuracy of the integrator. A first attempt is to use a 
midpoint method, which is related to Stratonovich inte¬ 
gration. During the time h, the error in the energy is 
given by 


AAi = V(xi) - V(a;i_i) -|- {xi - Xi-i) ■ F(Xi) (9) 

with Xi = {xi+i + Xi)l2. The modified OM functional, 
can be written in terms of the above and we get 




^ 1 

2 = 1 


Axi 


At 


1—2 


- ^ log(l + V"(A,) ) + AVi - AE, 


( 10 ) 


where the log term comes from the Jacobian that results 
from converting the Gaussian distribution of the variable 
^ to the distribution of x. The advantage of over 4) 
is that the energy error vanishes for an OU process. For 
nonlinear forces the AE terms do not vanish and may 
be significant in that it leads to spurious entropy pro¬ 
duction. These terms break time-reversal invariance or 
in the Monte-Garlo picture, they break detailed balance. 
The paths generated using the midpoint integrator will of 
be higher fidelity than those generate by the HMC with 
the leapfrog integrator, because the energy error and the 
spurious entropy production are smaller. 

With the midpoint method, we use the functional de¬ 
scribed in equation 10 with the path sampling scheme 


based on an HMG implementation using the ’’Implicit 
Algorithm” in Beskos et al. m As shown in Figure]^ we 
find that sampled paths are very similar to those based 
on integrating forward in time as described in equation 
In contrast to the results when using the Ito-Girsanov 
functional the value of B fluctuates around a value of 


0.8, about 10% smaller than its equilibrium value. Com¬ 
pared to a value close to zero found previously, this is a 
dramatic improvement. 


THE ZERO-TEMPERATURE LIMIT 

At zero temperature, one can consider the path that 
starts in one well, ends in an other, and transverses the 
barrier. Large Deviation theory[TB] coupled with the Ito- 
Girsanov functional provides a solution that corresponds 
to a heteroclinic orbit in an effective potential defined 
by Veff{x) = —|F(x)p/2. Such orbits take an infinite 
time. However, one can find approximate minimizers to 
the corresponding Euler-Lagrange equations. These ap¬ 
proximate minimizers are characterized by long periods 
of time spent at the critical points (including the maxi¬ 
mum, i.e. the barrier). 

At zero temperature such large-deviation solutions of 
the Ito-Girsanov functional are not physical[T^. When 
the temperature of the reservoir is zero, fluctuations van¬ 
ish. Thus this limit is also singular. Looking at se¬ 
quences of paths, using a nonzero time step with mono- 
tonically decreasing temperature, we find that the origi¬ 
nal OM functional ignores the energy errors, as expressed 
in equation!^ At very low temperatures, the Metropolis- 
Hastings rejection rate is very large. These errors dra¬ 
matically change the acceptance probability as the tem¬ 
perature approaches zero. Thus the fidelity of the path 
becomes increasingly compromised as temperature is de¬ 
creased. This is a direct consequence of the uncontrolled 
approximations inherent in Brownian dynamics. Thus 
as the temperature monotonically decreases, the large- 
deviation solution can not be the limit of the sequence 
of paths generated by equation]^ if one rejects states ac¬ 
cording to the Metropolis-Hastings criterion. 


DISCUSSION AND CONCLUSION 

The Onsager-Machlup functional is based on Brownian 
dynamics and provides a good starting point to under¬ 
stand the way a particle moves in response to thermo¬ 
dynamic fluctuations. However, it contains uncontrolled 
approximations. Here we used a novel approach based on 
the Metropolis Monte Garlo method to elucidate the er¬ 
rors and to show how the underlying measure will stray 
away from the correct thermodynamical one as either 
the continuous-time limit or the zero-temperature limit 
is taken. 

The consequences of the singular limit impacts a wide 
body of work. The theory for entropy production in 
nonequilibrium thermodynamics is just one of them. As 
an example see the work of Speck et al[2D]- For prob¬ 
ing the folding of proteins, the recent work of Fujisaki, 
et al[^ suffered from using the unphysical form of the 
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Onsager-Machlup function, while the older work of East¬ 
man et al|22j could be improved by using the functional 
based on the midpoint quadrature as this would decrease 
the spurious entropy production. Other works [53J [53] 
have to be reevaluated in light of this insight. 

The form of the OM functional has been explored 
before [2^2 7j . It is now clear why the issue of the form 
of the OM measure remained controversial for such a 
long time. Without an understanding of the limiting 
process, the answer remained hidden. In this paper, we 
described diffusive motion (Brownian dynamics) in terms 
of a Monte Carlo process. This allowed us to clearly ex¬ 
press the OM functional for small, nonzero time steps 
and to explicitly state the errors made in such a repre¬ 
sentation. Through the lens provided by the Monte Carlo 
procedure, we identified additional terms that contribute 
to the path probability. Thus we were able to uncover 
that the singular nature of the continuous-time limit of 
the OM measure lies in its very form. 

We end with a conjecture. How do we interpret the 
meaning of the Ito-Girsanov change of measure (equation 
[^? Consider the set of all paths, Hp-, generated from 
solving equation the index indicating the force. We 
speculate that in the continuous-time limit the sets Hi? 
and Hf^o are disjoint. Then equation simply gives 
an indication of the deviation of a free-Brownian path 
from a path generated when using a nonzero force. Thus 
the Ito-Girsanov change of measure is not related to the 
probability of the path. 
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